Cache memory in computer systems is used to store a copy of data items, be it either instructions or operands from main memory locations, for convenient and fast access by a central processing unit (CPU). Cache reuse is highly desirable as it reduces memory access time and thus, leads to significant execution speedup for most software applications. Efficient cache reuse within a process (or a thread in a multithreaded application) depends on the quality of the application program code, while inter-process and inter-thread efficient cache reuse involves keeping track of what has already been loaded in the cache and selecting subsequent processes/threads for execution next in view of cache content reuse.
Keeping track of cache content is achieved in the art typically by either employing hardware means or software techniques. Often, there is a trade-off between the accuracy of the cache content information gathered and the amount of time and resources dedicated to that process. Some cache reuse methods are generalised and thus applicable to any program or process, while others are more specialised.
Furthermore, some methods target data cache content reuse, while others target instruction cache content reuse. In some cases, the method is applicable to both data and instruction, without targeting either in particular.
One approach to data cache reuse known in the prior art is as follows. Processes that do not share any data are scheduled on different processors if possible in a multi-processor system. In addition the processes that cannot be executed at the same time due to dependencies, but do share common data among each other are scheduled on CPUs in a way that allows them to share cache content. The method requires detailed knowledge of the algorithms operative in the particular application and only data reuse is considered.
Other prior art aims at increasing the instruction data locality by scheduling the same process on the same processor. Only instruction reuse is considered.
A main shortcoming of some prior art methods is that they are designed to either facilitate data reuse or instruction reuse, but do not consider both.
Other, more general prior art approaches facilitate both instruction and data reuse through either hardware means or software techniques by measuring the cache warmth of the cache content with regards to a given process. Cache warmth is a term sometimes used to describe the age of data of a specific process found in cache and is measured in various ways in prior art, including methods such as counting the number of requests of each processor for each process, or keeping track of the cache misses for subsections of the cache, or even tracking the cache use on a line-by-line basis. A cache miss occurs when the CPU cannot find a data item in the cache. This necessitates fetching the data item from either lower level caches or main memory with an associated performance penalty.
The main disadvantages of these more general prior art approaches to cache content reuse lie in their complexity and the overhead that complexity imposes on program execution. They also often require specialised hardware for their implementation.